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OPTIMIZATION  OF  LINEAR  DISCRETE  SYSTEMS: 
AN  APPROACH  TO  THE  STAIRCASE  PROBLEM 


We  are  concerned  with  the  study  of  the  optimization  of  a 
stationary  staircase  problem,  mixing  in  our  approach,  the  concepts 
of  mathematical  programming  and  optimal  control.   Some  results  are 
first  derived  about  feasibility  and  controllability  as  a  greater 
understanding  of  system  behavior.   We  then  look  at  the  optimization 
using  a  dynamic  programming  approach  to  show  that  we  can  theoretically 
reformulate  our  problem  as  a  decoupled  optimization  problem.   Suggestions 
are  made  to  take  advantage  of  this  fact  in  designing  practical 
algorithms. 


53587^- 


Introduction 

In  the  past  few  years,  there  have  been  many  new  developments  in  the 
fields  of  mathematical  programming  and  systems  theory  but  to  the  best  of 
our  knowledge,  very  few  attempts  have  been  made  to  link  the  two  fields. 
Some  connections  have  been  made  in  control  theory  (e.g.  Rosen,  Pearson, 
Van  Slyke).   We  should  also  quote  the  work  of  economists  who  have  followed 
the  initial  studies  by  Von  Neumann  for  growth  and  planning  models  (e.g.  Gale) 

On  the  other  hand  operations  research  is  more  and  more  concerned 
with  systems  analysis.   One  can  argue  that  the  essential  problems  lie  in 
the  search  for  the  relevant  factors  and  in  the  modelling  of  the  actual 
system.   But  is  is  also  clear  that  the  appropriate  mathematical  techniques 
have  not  been  yet  fully  developed. 

It  also  appears  that  many  studies  deal  with  static  optimization 
when  the  problem  should  be  considered  in  a  dynamic  environment.  If  we 
consider  for  Instance  the  optimization  of  a  production  system, the  actual 
decisions  that  we  are  going  to  take  will  in  fact  condition  the  future  by 
affecting  for  instance  the  inventory  level.  Even  if  we  consider  several 
periods  simultaneously,  we  are  bound  by  the  present  techniques  to  fix  a 
finite  horizon  and  the  same  problem  of  accounting  for  the  future  remains. 

We  can  think  of  using  some  pricing  devices  to  measure  the  value  of 
the  final  state  reached  by  the  system  and  considering  as  an  objective  the 
optimal  trade-off  between  the  yield  in  the  periods  considered  and  the  final 
value.   Another  possibility  is  to  figure  out,  a  priori,  a  target  set  of 
acceptable  final  states,  thus  introducing  supplementary  constraints  in 
the  problem. 


Neither  method  is  fully  satisfactory  since  both  require  some  guess'-^ork 
about  what  the  optimal  should  be.   In  the  first  method  the  right  prices 
to  introduce  would  be  the  dual  prices  in  the  infinite  horizon  problem. 
The  second  method  allows  for  undesirable  latitude  on  the  final  state. 

What  we  intend  to  do  instead  Is  to  study  directly  the  infinite 
horizon  case  to  eliminate   the  bias  introduced  by  the  previous  methods. 
This  can  be  done  at  the  important  cost,  acceptable  for  some  problems  but 
not  for  others,  of  assuming  enough  stationarity  to  be  able  to  derive 
formal  results.   Indeed,  this  assumption  does  not  surprise  anyone  aware 
of  the  kind  of  difficulties  that  one  meets  in  optimal  control. 

Justifications  can  easily  be  found  in  that  projections  in  the  long 
run  are  by  nature  very  smooth,  typically  the  output  of  an  exponential 
smoothing  model.   Therefore  the  exogenous  variables,  inputs  to  our  models, 
will  have  the  stationnarity  required.   Furthermore  it  is  likely  that  we 
do  not  expect  any  change  in  the  structure  of  our  model,  if  for  no  other 
reasons  than  thatwe  cannot  select  any  most  likely  change. 

The  goal  of  such  a  study  is  to  define  new  methodology  for  long  range 
planning  and  to  provide  a  guide  for  taking  short  or  medium  range  decisions 
which  would  affect  the  system  behaviour  in  the  way  judged  the  most  compatible 
with  the  present  views  on  the  future. 


Presentation  of  the  Model 

Although  most  of  the  concepts  that  we  will  introduce  could  be 
generalized,  we  will  only  deal  with  linear  models  in  this  paper,  thus 
assuming  constant  return  to  scale.  We  also  assume  discrete  time. 


At  the  beginning  of  each  period,  we  are  faced  with  the  problem  of  fixing 
the  level  of  available  activities  to  satisfy  some  constraints,  but  we 
assume  that  the  level  of  those  constraints  has  been  affected  by  the 
decisions  taken  for  the  K  previous  periods. 

This  can  be  formalized  as  choosing  at  time  nay  such  as  to  satisfy: 

n  =  1,2,... 
1  K 

"n^n  "  "^n  "  Vn-1  "  •••  °  Vn-K  U  V   (Q,P)  matrices 

n  n 

r  e  eQ 

(See  Veinott  for  a  practical  application  of  such  models. )yn  ^ 

If  we  assume  that  the  maximum  time  lag  is  bounded  for  any  period, 
then  we  can  introduce  artificial  variables  and  increase  the  dimension- 
ality to  reduce  the  equations  to  carry  on  only  two  periods 

n  =  1,2.... 

B  X  =  r   -  A  X  , 
n  n    n    n  n-i  t.   «   /■   \    ^  • 

B  ,A   (q,p)  matrices 
n  n 

Our  assumptions  of  stationarity  are 


A  =  A 
n 


B   =  B 


i.e.  no  change  in  technology 


and       r  =  p  r. 


i.e.  exponentially  growing  requirement,  for  instance.   Setting 
* 
n       A    * 
X  =  p  X  ,  A  =  — ,  B  =  B,  the  model  reduces  to  Ax    +  Bx  =  r  o 
n      n      p  n-i     n 


We  should  also  allow  for  constraints  on  the  level  of  our  activities 
(e.g.  positivity,  work  force  limitation  in  period  n) .   If  we  assume  that 
the  endogenous  variables  fixing  those  constraints  also  vary  at  the 
same  rate,  the  same  change  in  the  variable  X  will  reduce  those  con- 
straints to  a  stationnary  one  x  e  Ji,  where  n  will  be  in  general  a 
bounded  convex  set. 

Most  of  those  assumptions  would  make  sense  in  macro-economics  and 
our  model  can  be  interpreted  as  a  generalization  of  the  Van-Neumann  and 
Leon tie f  models. 

To  sum  up  we  will  be  concerned  in  this  paper  with  the  model 

n  =  1,2,... 

Ax  ,  +  Bx  =  r„  X  e  E^  ,  r  0  E  E^ 

n-1     n    0  n 

A,B   (q  X  p)  matrices 

X  e  n   convex  compact  of  E 

There  are  two  types  of  questions  that  one  can  ask  about  the 
model  which  will  be  considered  in  the  following  sections: 
a)   Study  of  the  feasibility  (sections  II  and  III) 

Given  the  equations  of  the  systems,  we  have  to  characterize 
the  set  of  feasible  initial  points,  i.e.  the  set  of  initial 
points  from  which  one  can  derive  an  infinite ~  sequence  of 
feasible  points  (a  feasible  program) . 

It  would  be  also  interesting  to  derive  a  concept  of 
controUaWlity  analogous  to  the  one  of  system  theory  to 
characterize  the  type  of  control  that  we  can  use  to  drive 
the  system. 


b)   Study  of  the  optimization  (sections  IV  and  V) 

How  can  we  control  the  system  to  optimize  some  objective 
criterion?   Since  we  consider  an  infinite  horizon  situation, 
we  shall  face  the  problem  of  the  convergence  of  infinite  sequences, 
and  this  will  require  the  introduction  of  a  number  of  new 
concepts. 

II.   Study  of  the  Feasibility 
1,   Given  the  model 


Ax   ,  +  Bx  =  r  A,B  e  LCE^.e'')         (q  x  p)  matrices 

n-1     n 


X  e  fi  X  e  £*■ 

n 


r  e  E^ 

n  =  0,1,2,... 


We  will  look  specifically  at  the  following  questions: 

Section  II. A:   Under  what  condition  does  there  exist  a  solution 

in  the  form  of  a  feasible  program  {x  }  for  n  =  0,...,<»  ? 

n 

Section  II. B:   Given  an  initial  point  x  ,  what  are  the  conditions 
for  an  infinite  sequence  {x  , . . . ,x^}  to  exist? 

Section  III:   We  will  look  to  some  numerical  example  to  clarify 
the  ideas  and  point  out  the  computational    difficulties.   We  will 
also  indicate  how  we  intend  to  proceed  further  and  conjecture  a 
number  of  formal  results. 


II. A.   Feasibility  of  the  System 

2.   Problem;  Does  there  exist  a  bounded  sequence  of  points 
belonging  to  Q   and  satisfying  the  equations  of  model  1. 

We  will  show  that  a  necessary  and  sufficient  condition  is  that 

there  exists  a  stationary  solution. 

Let  us  assume  that  there  exists  such  a  sequence  {y  }. 

n 

Since  it  is  bounded,  it  has  an  accumulation  point  Y  and  we  can 

extract  a  converging  subsequence  (yi^h  ^» 

Then  given  e  >  0,  N  such  that         „    My   -  y   I  I  •^  ^ 
^      p  nj^.n^  2.  N    ''■'n^   •'n2''  -  p 


n^.n.  e  K 


and  we  have,  letting  n  =  y   -  y 

S   "2    ^ 


Av   +  By     =  r 
'"l     "l+l 


Ay„   1  +  B<y  +  n   )  =  r 
2        1    o 


1     "2-^ 

Let       z   =  E    y 

e    n„  -  n       ^p 
p    2    1  p=n 


z   belongs  to  fJ  since  It  is  a  weighted  combination  points  of  U 
P 

Az   +  Bz   =  r  -  Bn 

e      e       n^  -  n,    £ 
P      P        2    1     p 


Letting  e  go  to  0,  we  find  a  sequence  {z   }  of  such  points  which  also 

has  an  accumulation  point  that  we  denote  by  Z. 

It  is  clear  that  Z  belongs  to  Q   as  limit  point  of  a  subsequence 


of  points  of  Q   and  that  Z  satisfies  (A+B)Z  =  r. 

Conversely  if  Z  belongs  to  fi  and  satisfies  (A+B)Z  =  r,  then  the 
sequence  (Z,Z...)  is  feasible. 

We  are  therefore  led  to  set  the  definition: 

3.  Definition  1;   The  system  is  said  to  be  feasible  iff 
there  exists  a  stationary  solution  belonging  to  ^. 

II. B.   Feasibility  of  the  Initial  State 

Assuming  that  at  time  0  the  system  is  in  state  x(0)  =  x.,  does 

there  exist  an  infinite  sequence  x   (n=0,l, .  . . ,p . . . )  satisfying  the 
equation  of  the  system? 

4.  Definition  2;   An  initial  state  x  is  said  to  be  feasible  if 
it  gives  birth  to  an  infinite  sequence. 

5.  Proposition  1:   The  set  of  feasible  initial  states  is  a  compact 
convex  denoted  by  C, 

The  convexity  is  clear.   We  could  show  the  closedness  directly 
by  successive  extraction  of  convergent  subsequences. 

A  more  instructive  proof  introduces  the  set  H  of  points  of  Q 
with  at  least  N  successors.   We  have  the  inclusion  ^m^m_i* 

6.  Definition  3;   Let  us  now  denote  T   the  multivalued  application 

r  :  X  ^-  {ylyefi.  Ax  +  By  =  r} 


7.   Lemma  1;  T       is  a  closed  application. 

Proof:   Let  F  be  a  closed  set  of  fi. 


Then    r  ""^(F)   =   {x|    yeF,   Ax  +  By  =   r,   xefi} 


Let  X  be  a  sequence  of  r   (F)  converging  to  x. 

The  sequence  y  where  y   is  any  point  in  r  (x  ) 
n        n  n 

has  an  accumulation  point  y  belonging  to  F  and 
such  that  y  e  r  (x) .   Therefore  x  belongs  to  fi. 

We  can  now  notice  that  r   (Q)   =  Q     and  the  set  of 
initial  feasible  points  is  nothing  else  than 

oo 

rt  i^»,  =  C.   This  is  the  intersection  of  an  infinity 

N 
N=l 

of  compact  sets.   The  intersection  in  non-empty  if 

the  system  is  feasible  since  at  least,  the  stationary 

solution  belongs  to  it.   Therefore  it  is  closed. 


8.  Proposition  2:      If  for  some  N  we  have  fi,  =  S2., , ,  then  C  =  ^  . 
*-— N     N+1  N 

Let  us  assume  that  x.  has  n  successors  x-,...,x  .   Belonging  to 
0  In 

J  it  belongs  to  ^, -i  »  and  there  exists  a  feasible  x  ^  satisfying  the 


constraints,  but  we  see  that  now  x,  e  i^  and  therefore  there  exists  an 
'  1    n 

X  ,„  and  by  induction  x  .   for  any  p. 

n+2      ■'  n+p 

9.   Remark ;   We  can  show  (see  Berge)  that  there  exists  a  maximum 

compact  set  included  in  C  =  H  ^  which  is  stable  by  F,  i.e.  there  exists 

n=l 

a  A  such  that  r(A)  =  A. 

Greater  insight  can  be  gained  by  expliciting  the  multivalued 
application  using  the  pseudo-inverse  notation  of  Penrose  (see  Penrose, 
Zadeh) . 

Let  us  recall  that,  given  a  matrix  A  with  real  coefficient,  there 

+ 
exists  a  unique  matrix  denoted  by  A  such  that: 


+     +         +  -+-  + 

AAA=A  AA=AA  A    (q,p)   matrix 

AA''"a  =  A  AA"^  =  a''"a 


where  A  denotes  the  transpose  of  A. 

AA  may  be  interpreted  as  a  projection  operator  in  the  space  E^ . 
Let  us  denote  by  E   the  stable  subspace  in  e".   Then  the  equation 
Ax  =  b  has  solutions  iff  b  belongs  to  E  since: 

Ax=AAAx=b=AAb 


and  then  the  general  solution  may  be  written  as: 


X  -  a"^  b+  (I  -  a'*'a)z 


where  z  is  any  vector  in  E'^  aodu  =  (I  -  A  A) Z  is  any  vector  such  that 
Bu  =  0.   Coming  back  to  our  problem,  we  can  write  T     as: 

r  :  X  ->  {y\ycQ,J-7.zE^   s.t.  y  =  B^(r-Ax)  +  (I-B  B)z} 

Letting  x  be  a  stationary  solution  of  (I),  we  can  also  write: 

r  :  X  ->  {x+yIx+yen,Jf.zEE"  s.t.  y  =  b'''Ax  +  (I-b'"b)z} 

These  results  hold  under: 

10.   Assumption  2:   The  matrix  B  contains  a(q  xq)regular  submatrix.  B" 

In  that  case  £_=£*',   To  understand  this,  notice  that  any  equation 
B 

Bx  =  c  has  at  least  one  solution   x_,=  (B')   c 

xg.  =  0 


10 


11.   Remark :   One  can  wonder  which  are  the  relations  between 
our  results  and  the  Kakutani  fixed  point  theorem  which  says  that  any 
upper  semi  continuous  application  of  a  convex  into  itself  has  a  fixed 
point.   One  of  the  elements  of  the  definition  of  the  upper  semi  continuity 
is  that  no  point  should  have  an  empty  image.   We  can  only  hope  to  apply 
this  theorem  to  the  restriction  of  T     to  the  set  C  =  A^-   A. 
meaningless  result,  indeed,  after  what  we  have  seen. 


111.   Numerical  Examples  and  Further  Directions  of  Research. 
12.   Example  A.   Let  us  consider  the  system: 


1       1       12 
4x  ,  +  3x  .,  +  4x  +  X  =21 
n-1     n-2     n    n 


0  <  X  <  4  (see  fig.  1) 

—  n  — 

0  <  x^  <  3 

—  n  — 


We  are  looking  for  the  set  C  of  initial  feasible  states, 

oo 

C  =f\   fi 

n 
n=l 

where  Q  is  defined  by  induction  as 
n 

Q  =  {xlxeQ,   Yen  ,  s.t.  Ax  +  BY  =  r} 
n     '         n-1 

12^=  {(x^,x^)|x^e[0,4],x^£[0,3]} 


> 

n/? 

\ 

^/3 

ill 

\ 

\ 

: 

A 

\ 

. 

^/^ 

k 

^  ^tl-AX 


%d.%        ^o    ^"-<*   ''''- 


t^^       ^ 


XU^       -^X-i^  J^ei-.    -      ^ 


11 


Q^    is  the  set  of  points  of  ^   which  have  at  least  one  successor  in  Q. 

Given    (x-,x  )  the  candidate  successors  lie  on  the  line 

12  12 

4x  +  X  =  21  -  4xq  -  3x_  which  intersects  Q   only  if  the  RHS 

1     2 
belongs  to  the  interval  [0,19],  i.e.  if  2  £  4x  +  3x  ^21.   We 

therefore  have 

Q^  =  n^r\    {(x^,x^)|2  <  4xJ  +  3Xq  <  21}. 

Similarly  n.  may  be  defined  as  the  intersection  of  fi  with  a 
narrower  band  such  that  any  point  of  f2„  has  an  image  in  fi^  .   Very 
generally 

\-%    n   ((x\x2)la^^4xj  +  3x2  <&^} 

and 

C  =  S^Q  n  {(xj.x^)  |a_^  ji  4xJ  +  3x^  <  QJ 


where 


a  =  lim  a 
n-+<» 


0  =  lim  B 
00       n 
n-^<=° 


We  can  verify  in  that  case  that  a^  =  — r 


(see  fig.  2) 


13.   Remark;   It  would  be  interesting  to  derive  systematic 
procedures  to  find  the  set  C  but  as  we  shall  see  in  the  next  example 
C  is  not  restrained  to  be  a  polyhedron  and  can  assume  a  variety  of 
different  shapes  which  makes  speculation  difficult. 


n. 


0-^ 


t<.o■'^^'^ 


h^ 


12 


14.   Example  B;   (see  fig.  3) 

In  this  example  x  is  completely  determined  by  x  _^  and  is 
obtained  by  a  positive  rotation  around  the  fixed  point  in  the  positive 
quadrant.  This  is  a  linear  transform,  in  accordance  with  our  general  model, 

We  define  f2  as  the  positive  quadrant. 

It  is  easy  to  realize  that  the  shape  of  C  is  a  regular  polyhedron 
if  the  angle  of  the  rotation  is  in  the  set  iitj  (i.e.  —  is  rational)  or 
a  circle  if  it  is  not. 


Some  Relevant  Concepts; 

15.   Controllability;   we  will  temporarily  forget  about  the 
constraint  x  e  fl  and  simply  study  the  system  defined  by 


Ax  ,  4-  Bx  =  r 
n-1     n 


We  assume  that  B  contains  a  regular  square  submatrix.   We  have  seen 
that  in  that  case  we  could  write 


X  =  B'''(r  -  Ax^_^)  +  (I  -  b'''b)z 

where  z  is  an  arbitrary  vector  in  R^ .   Given  x  let  us  define  by 
recurrence 


^n  =  ^  ^'^  -  ^^-1^' 


The  general  solution  x  may  be  written  as 

X  =  x„  +  I      (-B  A)^(I  -  B  B)z 
^    ^    q=0  ^ 


13 


16.      Proposition;      If   the  matrix 


[(I  -  b'^b) : (-ba"^) (I  -  b"*"b):  :(b~a)P~-'-(i  -  b'^'b)] 


has  full  rank  p  then  we  can  drive  the  system  to  take  any  value  that 
we  wish  in  a  finite  number  of  steps. 

Proof:   It  is  obvious  in  that  case  that  the  vectors 

p-1    ,     ,      , 

I      (-B  A)P"-^(I  -  B  B)z 
q=0  "^ 

span  the  whole  set . 

17.  Remark;   The  rank  of  the  considered  matrix  is  of  prime 
importance.   Indeed  for  any  n  larger  than  p-1,  (-B  A)    may  be 
expressed  as  a  linear  combination  of  the  matrices  (-B  A)    q=l»  p-1  by 
the  Cai ley -Hamilton  theorem,  and  we  do  not  have  to  consider  more  than 
p  steps  to  obtain  the  maximum  flexibility. 

18.  Proposition  2:   For  n  larger  than  p,  and  fixed  x^,  x  is 
arbitrary  in  a  linear  variety  of  dimension  equal  to  the  rank  of  the 
considered  matrix. 

19.  Remark:  If  B  is  square  and  inversible,  the  problem  is 
deterministic  but  we  also  have  B  =  B  and  the  matrix  is  0.  We 
have  no  controllability  at  all. 

20.   Controllability  with  Supplementary  Constraints. 

Coming  back  to  our  first  example  we  can  study  what  flexibility 

1   2 
we  have  in  the  choice  of  successors.   Given  (x^,x-)  the  set  of  possible 


% 


'(ij 


r^to) 


r^Cx.) 


lA 

1    2 
successors  is  the  intersection  of  c  with  the  line  4x  +  x  = 

1     2 
21  -  4x^  -  3x  ,  which  is  noir  voiA  by  definition  of  C.   For  any 

point  other  than  A,  we  therefore  find  a  closed  segment.   By  iteration 

we  find  that  apart  from  A  we  get  from  any  point  to  any  point  in  a 

1  2 
band  of  the  type  y   <_  Ux  +x.   <_  &    in  two  steps. 

This  band  has  full  dimensionality  2  reflecting  the  fact  that 

in  absence  of  the  constraint  x     e   Q     the  system  would  be  completely 

n    n 

controllable. 

When  the  number  of  steps  increases  the  band  widens      up  to 

1    2 
the  point  where  it  actually  covers  the  maximum  range  5  <^  4x  +  x  <^  16. 

The  final  set  is  in  fact  smaller  than  the  original  C  but  is  accessible 

from  any  point  of  C.   This  set  is  the  maximum  stable  set  A  by  the 

operator  r  (of  remark  9),  A  =  H  r  (c) . 

n=l 
We  conjecture  that  if  the  system  is  fully  controllable,  in  the  sense 

that  we  have  given,  then  for  any  interior  point  of  C,  r  (x)  ->•  A  in  the 

Haussdorf  topology  for  compact  sets.   If  the  system  were  not 

controllable  then  r"(x)  is  the  intersection  of  A  with  a  linear  variety 

of  dimension  smaller  than  p. 


21.   Constructive  Definition  of  C. 

We  a  priori  know  that  C  contains  at  least  the  fixed  points  of  the 
mapping  r»   Let  F„  =  {x|xeS^.  ;/ (A  +  B)x  =  r)  ,  the  set  of  fixed  points 


of  r. 

We  can  now  work  backward  and  find  the  set  F  of     points 
which  have  an  image  in  F^. 

Inductively  we  define  F  =  {x|  xefi,3  yeF^_j^  s.t.  Ax  +  By  =  r}, 
Clearly  F  is  closed  and  compact  and  we  have  F  ID  ^n-l* 
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22.   Problem:   Do  we  have  C/  F  =  C? 

n 

We  shall  see  by  an  example  that  it  is  not  true  in  general. 
Consider  a  deterministic  two-dimensional  case,  where  the  operator  r 
is  a  geometrical  transform  such  that  the  transform  of  M.  is  given  by 
pM  =  k  pM  with  respect  to  a  fixed  point  P  then 


but 


and 


C  =  fi 


Fq  =  (PI 


F-  =  ...F  =  {P} 
1       n 


23.   Remark;   We  nevertheless  conjecture  that  the  following  will  hold: 
■if  the  system  is  fully  controllable,  then 


U  F  =  C 

n 
n 


This  idea  would  lead  to  the  definition  of  a  computational  check  to 
find  whether  or  not  the  given  initial  point  is  actually  feasible. 


IV.   OPTIMIZATION  OF  THE  SYSTEM 
IV. A.   The  Problems. 

One  is  hardly  objective  in  the  choice  of  a  return  function.   In  a 
case  like  this,  where  mathematical  difficulties  are  met,  one  is  tempted 
to  choose  the  return  function  in  such  a  way  that  everything  goes  well. 
There  is  a  duality  between  the  return  function  and  the  optimal  program. 
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It  is  perfectly  likely  that  one  has  a  better  chance  to  determine 
a  priori  an  optimal  program  and  then  to  fix  the  return  function  to 
get  the  right  result.   This  approach  even  seems  to  be  the  most 
effective  one  to  guide  a  system  randomly  perturbed. 

Depending  on  the  problem  the  prior  information  will  be  either  an 
optimal  behaviour  (desired  but  not  necessarily  obtainable)  or  an 
objective  function.   In  the  former  case,  one  can  formulate  an  objective 
function  if  some  measure  of  the  proximity  of  two  programs  is  also  given, 
possibly  of  the  form  of  a  loss  function. 

Our  first  goal  will  be  to  define  the  infinite  horizon  solution  or 
optimal  long  run  behaviour  and  then  we  will  try  to  derive  some  turnpike- 
like theorems  for  the  case  where  one  starts  from  the  initial  state  x_. 

IV. B.   Optimality. 

Assuming  that  we  are  given  an  objective  function  separable  over 
time,  we  have  to  deal  with  infinite  sequences.   Two  alternatives  are  open. 

1)  Introduce  a  discount  factor  such  that  any  considered  sequence 
will  converge. 

2)  Use  the  criteria  of  optimality  suggested  by  Gale. 

* 

a)  A  program  x  is  said  to  be  strongly  optimal  if  it  overtakes 

any  other  program,  i.e.  for  the  minimization  case 

N  *  * 

Z[c(x)-c(x)]>0        V   N>N 
t^   n  n     n  n   —  — 

* 

b)  A  program  x  is  said  to  be  optimal  if  it  catches  up       any  othe 

program,  i.e.  in  the  minimization  case 
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V  e>0,3T    s.t. 


E[c  (x^)  -  c  (x^)]  >  -  e      N  >  T 
.nn     nn   —  —  e 


When  dealing  with  such  infinite  horizon  program,  one  has  to  be 
very  careful  for  there  does  not  necessarily  exist  an  optimal  program. 
A  very  good  example  of  this  phenomenon  is  provided  by  Gale  with  the 
sharing  of  a  pie  and  a  utility  function  c  =  c  which  is  convex.   Let 


e.  =  (  ^'  7.-..Ao....,o,...) 

n     n  n     n 
n  times 


The  sequence  (e  }   gives  an  increasing  return  and  nevertheless,  for 
z 

the  uniform  convergence  topology  converges  toward  (0, . . . ,0, . . . )  which  is 

the  worse  solution.   Furthermore  any  solution  can  be  overtaken.   Our 

feeling  is  that  those  problems  require  a  very  careful  mathematical 

analysis. 

IV. C.   Dynamic  Programming  Approach. 

Let  us  consider  the  stationary  model 

Ax  ,  +  Bx  =  r 
n-1     n 

x(0)  =  Xq 

X  e  n 
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A  strategy  will  consist  of  determining  a  priori  where  to  go  next  in 

the  form  of  a  sequence  of  function  (d  , ...,d  ,...).   One  will  then  take 

iteratively  x  ,,  =  d  (x  ) . 
n+i    n  n 

n 

A  strategy  will  be  feasible  from  x  if  7n  n   d   [x  ]  e  fi 

p=0  P   "     ^ 


implying  in  fact  that   II   d   [x.]  e  C    .A  strategy  will  be  said 
p=0  P   ° 

to  be   feasible  if  the  same  results  holds  for  any  initial  x  belonging 
to  the  set  C  of  feasible  initial  states. 

To  a  strategy  will  be  associated  a  return  specified  by  the  value 
of  some  objective  function.   Although  the  return  could  be  infinite,  it 
will  in  general  be  possible  to  compare  two  strategies  by  looking  at 
the  difference  in  the  return. 

In  a  stationary  problem  of  the  type  that  we  consider  it  is  likely 

that  the  objective  will  also  be  stationary  in  such  a  way  that  the  decision 

of       where  to  go  next  depends  only  from  the  state  and  not  from  the 

time  when  we  arrived  at  this  state.   Such  a  stationarity  may  be  taken 

into  account  by  an  objective  function  of  the  type  T.a   f  (x  )  and  occurs 

n=0    " 

indeed  only  for  objective  functions  of  this  type  when  the  further 
assumption  of  additive  separability  with  respect  to  time  is  added. 
f(x)  denotes  a  fixed  cost  structure,  a  is  a  discount  factor  that  could 
be  taken  to  unity.   In  that  case  it  should  be  clear  that  we  can  find 
a  stationary  infinite  horizon  strategy  of  the  form  [i}i  , .  .  .  ,<t> ,  . . .  ]   where 
<J|  is  a  function  in  C  with  value  in  C  (see  Blackwell  for  a  similar  state- 
ment for  discrete  dynamic  programming). 

We  will  follow  the  line  suggested  by  Denardo  to  clarify  this  approach. 
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Total  Return  Associated  with  a  Strategy 

Given  a  feasible  strategy   ,  i.e.  a  function  of  C  into  C,  the 
total  return  from  an  Initial  state  x  is  easily  found  as  the  discounted 
sum  of  the  successive  elementary  return.  Denoting  the  total  return 
by  g  we  get 

g  (x)  =  E  a"fo4."(x) 
'      n=0 

25.  Assumption;   If   we  assume  a  less  than  1  and  f  bounded  in 
C  for  things  to  converge  this  expression  can  be  written 

00 

g.(x)  =  f(x)  +  E  c.'^'^^of^-^C^Cx)) 

*  n=l 

or 

g^(x)  =  f(x)  +  g^<>>(x) 

26.  Proposition;   The  return  function  associated  with  a  stationary 
strategy   is  a  solution  of  the  recursive  equation 

T  =  f  +  To<J> 


Conversely,  what  could  we  say  about  this  recursive  equation? 

We  will  see  that  we  can  start  from  any  T_  and  build  the  sequence 

^n  =  ^  -^  Vl°  * 
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This  sequence  converges  toward  a  function  which  is  the  unique  solution 

of  the  equation  26.   And  we  will  have  simultaneously  shown  that  the 

solution  is  unique  and  given  a  construction  to  build  it.   Let  us 

formalize  this  idea  a  bit.   On  the  set  B(C,R)  of  bounded  function  C 

in  R  we  can  introduce  a  metric  p(T-,T^)  =  supJT^Cx)  -  T^(x)|.   Let  us 

xeC 

now  consider   the  mapping  f^,    of   B(C,R)    into   itself 
M.(T)   =   f  +  aTo<> 

27.      Proposition;      For   4>   less   than  1   the  mapping  M.  (T)    is 

contracting.      In  other  words,   given  T,  ,T-  we  have   p(>I  T^.M.T.)    <   p(T-,T„) 

12  ip    I      (f    I  1      Z 


Indeed  "^(T^)   -  M  (T^)   =  a(Tj0  4)  "  '^2°'^^ 


sup{M^(T^)    -  M^(T.)[x]}=   a  sup(  (T   -T-)o  <})  )  [x  ] 

/-.<pi           92  „       i     Z 

xeC      ^                   ^  xeC 

=  a  sup    {T^-T^^ly] 


ye*(c)- 

<_  a   sup    (T  -T    )[x] 
x<C 


PVrV2^      1«P(T,.T2) 


28.      Corollary   1:      The   solution  of   the  equation  26   is   unique. 
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If  not,  let  T,  and  T.  be  two  solutions.   The  solutions  satisfy  M,(T)  =  T 

1        Z  9 


therefore  we  would  have 


p(M^(T^),M^(T2))  =  pd^.T^)  <  apd^.T^) 


which  is  impossible. 

29.  Corollary  2;   The  unique  solution  may  be  found  by  successive 
approximations . 

Starting  from  some  bounded  T-.,  we  build  a  Cauchy  sequence  of  bounded 
functions  for  the  uniform  convergence  topology.  The  sequence  has  therefore 
a  limit  which  is  also  bounded. 

30.  Corollary  3;   If  f  is  continuous  and  4"  Is  continuous  then  the 
return  function  is  also  continuous,  since  it  is  obtained  as  uniform 
limit  of  continuous  function  if  one  starts  with  a  continous  T_. 

31.  Remark ;   a  =  1  does  not  mean  that  the  sequence  T  is  not 
convergent,  but  one  would  have  to  look  explicitly  if  the  particular  (Jj 
chosen  implies  a  contraction. 

32.  Remark:   We  have  g,  =  lira  M"(g)  for  any  initial  g.   This 

(J,  !p 

n-xxi 
convergence  may  be  made  more  precise  if  one  notices  that 

l|M^(g)  -  g||  >  llg^  -  slid  -  «)• 

33.  Definition;   We  will  define  as  optimal  return  the  function 
g(x)  =  sup  g.(x) 

This  sup  exists  whenever  F  is  bounded  and  a  is  less  than  1. 
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Updating  the  Expected  Return 

Assume  that  we  make  a  guess  at  the  optimal  return  from  any  state 
of  C  of  the  form  of  a  function  g(x).   Assume  that  we  are  in  state  x. 
With  this  prior  knowledge  we  would  consider  the  one  stage  optimization 
problem: 


find  y  such  that    r    max  g(y)  =  g(y„) 
Ax  +  By  =  r 
y  e  C 


in  order  to  go  to  the  most  promising  state  among  those  accessible  from  x. 

But  the  expected  return  from  x  is  no  longer  g(x)  but  f(x)+a  aup   g(y) 

ye  r(x) 

and  we  have  in  a  way  updated  our  expected  return.   This  idea  leads  us  to 

consider  the  mapping  denoted  by  A  defined  from  B(C,R)  into  itself. 

34.       Ag(x)  =  f(x)  +  sup   {g(y)} 

ye  r(x) 

where  r(x)  as  in  section  II  denotes  the  set  of  acceptable  successors  of  x. 

35.  Proposition;   If  a  <  1  the  mapping  A  is  contracting  (see 
Denardo  for  a  proof) . 

36.  Corollary  1:   The  mapping  A  has  a  unique  fixed  point  denoted 
t>y  g  »  solution  of  the  recursive  equation 


;(x)  =  f(x)  +  a  sup   g(y) 
ye  r(x) 


* 
37,   Corollary  2;   The  function  g  may  be  found  by  successive 

approximations  as  lim  A  g  for  any  Initial  bounded  g. 
n^^ 
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These  two  corollaries  are  proved  as  29  and  30.   Before  going  further  and 
giving  the  properties  of  g  ,  such  as  continuity,  convexity,  etc.,  we 
will  see  the  relation  between  g  ,  the  updated  return,  and  g  the 
optimal  return  defined  in  33. 

38.  Proposition;   For  any  E  arbitrarily  small,  there  exists  a 

strategy  ((i  whose  associated  return  function  g  is  e-close  of  g  ,  i.e. 
* 

39.  Proposition;   For  a  <  1  the  optimal  return  g  is  in  fact  equal 

* 
to  g  . 


-By  Proposition  38, g  is  approximated  as  close  as  we  wish  by  some 

* 
g  therefore  g  £  g. 

-On  the  other  hand  for  any  (fi 

iteratively 

M^(g  )  1  %   (g  )  1  ...  1  g 

at  the  limit 


g  =  lim  M"(g*)  <  g* 


and  therefore 
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* 
g(x)  =  sup  g.  (x)  £  g  (x)  for  any  x. 

* 
By  combination,  we  deduce  g  =  g  . 

Optimal  Return  and  Optimal  Strategy 

From  the  previous  results  we  have  seen  that  there  exists  an  optimal 

* 
return  function  g  whenever  f  is  bounded  as  the  set  C  of  initial  feasible 

solution  and  we  apply  a  discount  factor  a  <  1. 

This  function  g  is  obtained  as  the  unique  fixed  point  of  the 

* 
mapping  Ag  =  f (x)  +  a  sup    g(y)  or  equlvalently  g  is  the  unique 

ye  Ik 
solution  of  the  recursive  equation 


g  (x)=  f(x)  +  a  sup    g  (y) 

ye  Tx 


We  have  also  pointed  out  that  there  exists  a  strategy  such  that 
the  corresponding  return  function  is  arbitrarily  close  to  g  . 
40.  Assumption:   In  the  following  we  will  assume 


^ C  compact 

I  f  continuous. 


* 

41.  Proposition;   g  is  continuous. 


Proof;   g  =  lim  A  g^  for  any  g^  for  the  uniform  topology. 


-but  if  g  is  continuous  then  A  is  continuous.   Indeed 


Ag(x)  =  f(x)  +  sup  g(y) 
ye  Tx. 
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the  optimal  return   h(b)  of  the  parametric  program. 

sup  g(y)  =  h(b) 

By  =  b 

y  e  C 

is  a  continuous  function  of  b.   If  we  set  b  =  r  -  Ax 
it  is  also  a  continuous  function  of  x  (the  program  is 
consistent  whenever  x  e  C) . 

g  obtained  as  uniform  limit  of  continuous  function, is 
continuous. 
Proposition;   There  exists  an  optimal  strategy. 


Proof:   Given  g  ,  let  us  consider  the  parametric  program 
sup  g  (y) 
Ax  +  By  =  r 
X  e  C 
y  e  C 
The  set  of  feasible  y  is  r(x) ,  the  intersection  of  a 

linear  variety  and  a  compact  non-empty  set  by  definition^ 

* 
is  compact.   The  function  g  (y)  obtains  its  maximum  at 

* 

a  point  y  (x) . 

The  function  (|>;x  ->■  y  (x)  is  a  strategy  whose  return  is 
the  unique  solution  of,  the  recursive  equation 


g(x)  =  f(x)  +  ag)<|)(x)  =  f(x)  +  a   sup    g(y) 

ye  Vx. 
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* 
which  is  the  same  equation  as  the  one  vhich  defines  g  . 

43.   Theorem  of  Separability 

The  solution  of  the  linear  program 

max  ^a   f(x^) 

where  a  is  a  discount  factor 

,  „  less  than  1 

Ax  ,  +  Bx  =  r 
n-1     n 

X  e  n 
0 

may  be  found  by  solving  a  succession  of  decomposed  programs 

* 

max  g  (x  ) 
n 

Ax   1  +  Bx  =  r 
n-1     n 

X  e  C 
n 

where  C  denotes  the  set  of  initial  feasible  points  C  =  H  ^ 

i£0_:: 

Q     =  {x|xeC,  X  has  at  least  n  successors  in  ^J   and  g  is  the 
optimal  return  function. 

44.   Proposition;    if  f  is  convex  the  optimal  return  function  g  is  convex. 


Proof:   Let  <i>   be  an  associate  strategy 
Let  XQ.y^  £  C 

e  e  [0,1] 


The  sequence  gifi  (x„)  +  (1  -  B)(J)  (y-)  is  feasible 
starting  from  the  point  gx.  +  (1  -  B)yQ. 
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The  return  of  the  corresponding  strategy  is  given  by 

j:a"f(B<^''(xQ)  +  (l-6)<^"(yQ))  >  eg*(xQ)  +  (l-e)g*  (x^^) . 

Furthermore  the  optimal  strategy  (j)  from  Bx  +  (l-6)y^ 
is  at  least  as  good  as  the  one  we  have  explicited. 
We  therefore  get 

g*(BxQ  +  (l-B)yQ))  1  Z^a''f(64)"(xQ)  +  (l-6)*"(yQ)) 

Combining  the  two  inequalities  gives  the  desired  result. 

45.   Remark :   If  f  is  convex  one  can  expect  an  interesting  result. 
In  particular,  we  postulate  that  one  solution  of  the  parametric  program 

* 
max  g  (y) 

Ax  +  By  =  r 

X  e  C 

may  be  found  as  a  continuous  function  of  x  and  therefore  we  should  look 
for  a  continuous  optimal  strategy. 

V.   NUMERICAL  EXAMPLE  OF  OPTIMIZATION 


46.   Problem:   E  h{^^   +  y^) 


i  1  +  y  ,  =  2x  +  3y 
n-1   -^n-l     n    -^n 


We  look  for  the  return  first  and  the  optimal  strategy.   By  application 
of  the  Kuhn-Tucker  Theorem 

^''^Pn-  Pn+1  =  °  "  =  ^ " 

^n  -^  3Pn  -  Pn+1  =  « 

Vl-'  Vl  =  ^^n-'  ^^n        "  =  ^••••'" 
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where  p  denotes  the  price  associated  with  the  n*-  equation.   Eliminating 

X  and  y  we  get 

n     -^n    ^ 

Pn+l-  ^Pn-'Pn-l^  °  "  =  ^ " 


The  general  solution  of  this  equation  may  be  written  as 


p  =  Xa    +  \ip 


2 
where  a  and  g  are  the  two  roots  of  the  equation  p  -  3p  +  1  =  0. 

Generalizing  the  transversality  conditions  of  the  minimum  principle, 

we  should  find  lim  p  =0.   Since  only  one  root  has  modules  less  than  1, 
'^n 

that  gives  an  expression  of  p 

,  n-1  ,^,      3  -  /5         , 

p   =  Xa    with  a  =  — ■;; n  >  1  . 

'^n  2        = 


We  shall  fix  X  be  the  initial  conditions.   We  have 


Xj^  =  P2  -  2pj^  =  X(o-2) 


y^  =  P2  -  2p^  =  X(a-3) 


^0  ■"  ^0  =  2^1  ^  ^^1 


=  X(5a-13) 
^  "  5ci  -  13 
and  we  finally  get 


„     ,  n-l,   „.    7  -  3v^  ^  n-1,   ,   s 
'^n  =  Pn+1  -  2Pn  =  ^^        (""2)  =  — ^ ^  «    ^V^O^ 
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,^JU\ 


c,>^V^c^^  ^ 


\,*^VN«-         •^ 


^V.lH^  /)-^c^ss< 


fcc^  r 
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^n  =  Pn+1  -  3Pn  =  ^°'"'  ^^''^^  =  <^  "  ^^'^^^^^o'-^O^ 


The  successive  points  always  lie  on  the  line 


Z   = 


3  -  a   1  +  >^ 


X   2  -  a     2 


One  should  notice  that  given  a  point  (x^,y-)  the  next  point  is  to  be 
taken  somewhere  on  the  line  2x  +  3y  =  x  +  y  . 

A  tradeoff  has  to  be  realized  between  the  one-step  optimization 
which  would  consist  if  taking  the  projection  P  of  0  and  the  point  Q 
which  would  completely  discount  the  future  at  a  lower  cost  in  the 
present.   The  return  function  may  be  found  a  posteriori  as 

g  (x,y)  =  h  l(x   ^  +  y   ^)   =  hUf,   +  y^)  +  h   Z  (x^  +  y^) 
n     n        U    U         _    _ 


=  4  (Xq  +  y^)  +  % ^  X  I  _^—   +  (/5-2)  I 

1  -  a     ^  J 


One  can  verify  that  the  return  function  is  also  quadratic  definite 
positive. 

50.   Remark:   Give  a  point  x  ,y„,  one  has  to  choose  its  image  on 
the  line  2x  +  3y  =  x  +  y^,  by  finding  on  this  line  the  point  which 
minimizes  the  quadratic  form.   This  is  easily  found  by  taking  the 
direction  conjugate  of  (3,-2)  and  drawing  the  corresponding  line  by 
the  origin.   The  intersection  point  is  the  point  sought. 

We  could  verify  in  this  case  that  the  direction  (3,-2)  and 

,,   1+^T. 

(1,  — — )  are  conjugate. 
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Direct  Solution  for  the  Return  Function 

We  shall  start  by  noting  that 

-the  future  process  depends  on  x    ,y^   only  through  x  +  y  . 

-if  the  objective  is  homogenous,   the  return  function  will  also 
be  homogenous  to  some  degree. 
We  are  immediately  led  to  look  for  a  return  function  of  the  type 


S(x,y)  "   '-^^^^^y^)  +  A(x+y)^ 


We  are  left  with  the  problem  of  determining  A  to  satisfy  the 
recurrence  equation 

51.       g*(x.y)  =  hi^^+y^)  +   sup    {h(.x^-^^)  +   A(x+Y)^} 

(X,Y)er 

while 

r  =  {(X,Y)|2X  +  3y  =  X  +  y} 

To  solve  the  problem  of  finding  the  maximum  we  just  have  to  find  the 
conjugate  direction  of  (-3,2)  for  the  quadratic  form  g  . 

We  get  the  direction  (5X+1,  5X+  3/2). 

We  then  solve  the  system     2x  +  3y  =  x  +  y 


5X+1   5A+  3/2 

Identification  of  both   sides  of  equation  48  wil  then  give  the  value 
of  X. 
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VI.   FURTHER  PERSPECTIVES 

The  general  linear  model  that  we  have  been  considering  up  to  now 
is  fairly  general  and  we   think  that  it  could  be  relevant 

in  planning  in  a  number  of  cases.   We  have  been  assuming  that  we  would 
be    able  to  provide  an  objective  function  and  that  is  a  debatable 
occurrence.   What  we  want  to  suggest  is  that  it  could  be  more 
reasonable  in  practice  to  adapt  a  two  stage  optimization  of  the 
following  type. 

Step  1;   Assuming  that  the  system  is  in  a  steady  state,  what 

would  be  the  best  solution?   This  is  equivalent  to  solving 
the  one-step  problem 

maximize  return  or  minimize  cost 
given  (A+B)x  =  r 
X  e  fi 
Step  2;   In  general,  at  the  time  when  we  start  the  initial  state 
X-  is  different  from  the  optimum  steady  state  x.   It 
seems  easier  at  that  point  to  find  a  loss  function,  measuring 
what  is  lost  compared  to  x.   As   in     decision  theory 
the  most  practical  case  will  be  to  consider  a  quadratic  loss 
function. 
This  technique  presents  several  advantages. 

-the  return  function  as  defined  previously  will  be  in  general 
finite  even  if  we  do  not  consider  discounted  cost.   Indeed  if  the  system 
is  controllable  (see  23)  we  can  get  to  x  in  a  finite  number  of  steps. 
We  can  exhibit  a  program  with  finite  value.   Most  of  the  result  of 
section  IV  will  hold  even  if  a  =  1. 

-the  other  practical  alternative  is  to  consider  linear  cost.   It  is  possib 
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to  show  that  whenever  we  Introduce  a  discount  factor  in  that  case,  the 

optimal  solution  will  in  general  tend  to  be  periodic  and  not   steady 

state.   This  type  of  solution  would  be  difficult  to  justify  in  any 

practical  application. 

-It  is  more  likely  to  be  workable  as  seen  in  the  example  of 

section  V.   In  absence  of  the  constraint  x  e  fi  we  can  ascertain  that 

n 

-the  return  function  is  a  quadratic  form 

-it  can  be  found  by  iteration  (see  30)  and  this  is  tractable  since 
one  will  work  only  with  the  quadratic  form,  computationally  stored  in 
matrix  form. 

Sketch  of  an  Algorithm  for  a  Quadratic  Case 
Quadratic  Optimization 

53.   Given  the  problem 


min  h(Y^    ) 
Ax  +  By  =  0 


We  can  apply  the  Kuhn-Tucker  Theorem  and  we  should  find  y  and  p  such  that 


Let 


-1 


Py  +  B  p  =  0 
By       =  -Ax 


"■[:  :■] 


"^11     *12 
*21     *22 
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54.   We  get  y  =  -^  jAx 


Optimal  Return 

Given  the  problem 


n  n 


Ax  ,+  Bx   =0 
n-1    n 


We  can  find  the  optimal  return  in  the  form  x  K.x  where  K  satisfies 
the  recursive  equation 


55.       K  =  Q  +  A.\^^0!i)Ki>^2(   K)A 


We  can  find  this  solution  as  the  limit  of  Q 


56.       Qn  =  Q-^^M2(Vl>Qn-l*12^Vl>^ 


where  one  would  take  for  instance  0  =  Q  in  the  absence  of  other 
information. 


34 

Relations  with  Control  Theory 

In  the  second  part  of  this  paper  we  have  shown  the  existence  of  an 
optimal  return  function  satisfying  the  recursive  equation 


g  =  f  +  sup  (g(y)) 
ye  Pjc 


This  equation  can  be  viewed  as  a  particularization  to  the  discrete  case  of 
the  Hamilton-Jacobi-Bellman  equation 


*  * 

1^  +  sup  {L(x,u)  +  <  If  ,  f(x,u)>} 


characterizing  the  optimal  cost  to  go  g  (x,t)  for  the  control  problem: 
find  u  which  maximizes  the  cost  of  L(x,u)dt 
subject  to  x  =  f(x,u) 

We  have  pointed  out  conditions  of  existence  of  a  solution  of  this 
equation  and  derived  a  way  to  actually  find  it.   In  this  light,  the  comments 
on  the  quadratic  case  are  parallel  to  the  solution  of  the  classical  tracking 
problem 

min  li(x*^Qx  +  u'^Ru) 

Quite  similarly  the  cost  to  go  can  be  be  expressed  as  a  quadratic  form 

*         t 
g  (x,t)  =  X  K(t)x  when  the  horizon  goes  to  infinity,  the  cost  is  independent 

*  t 

of  the  time  and  we  find  g  as  a  quadratic  form  ijx  Kx  where  K  is  the  unique 

positive  definite  matrix,  stationary  solution  of  the  Riccatti  equation 
K  =  -KA  -  A^K  +  KBR~-''B'K  -  Q 


35 


K  A  +  A^K  =  Q  -  KBR~''"B'K 


to  be  compared  with  our  equation 


K  =  Q  +  aSJ2^^^*^'''i2^^^^  ^^^^ 


Despite  the  similarities  our  problem  is  not  a  classical  control 
problem  for  two  reasons; 

-There  is  no  clear  cut  distinction  between  state  variables  and 

control  variables.   We  could  distinguish  between 

(x, , . . . ,x  )  as  state  variables 
1     q 

(x  ,1 X  )  as  control  variables 

q+1      p 

since  the  choice  of  the  latter  will  fix  the  former.   But  this  is  indeed 
very  artificial. 

-There  are    constraints  on  the  set  of  controls  usable  at  each  point 
in  time. 


Conclusion 

Throughout  the  paper,  we  have  mentioned  conjectures  and  new  problems 
suggested  by  this  approach  of  the  staircase  optimization  problem.   This  is 
enough  to  justify  the  interest  of  the  mathematician. 

As  operations  researchers,  our  attitude  is  somewhat  different  and 
much  more  application  oriented.   We  think  that  there  is  a  good  chance  at 
this  point  that  we  can  end  up  with  some  efficient  algorithm  in  the 
quadratic  formulation. 
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We  suspect  the  model  to  be  general  enough  to  be  applied  in  a  number 
of  fields.   Imbedded  in  it  are  models  by  Veinott  in  production,  Leontief  in 
economics  and  a  large  part  of  the  planning  model  of  Eckaus .   We  have  not 
at  this  point  specialized  in  any  one  of  those  applications  but  we  think  that 
the  general  result  that  we  obtained  could  give  more  insight  into  the 
behavior  and  the  control  of  the  dynamics  of  some  of  the  previous  models. 

Future  research  will  tend  to  be  more  complete  characterization  of 
problems  and  algorithms  for  solving  them. 
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